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Abstract 


Greedy algorithms exploit problem structure and constraints to achieve linear-time performance. 
Yet there is still no completely satisfactory way of constructing greedy algorithms. For example, the 
Greedy Algorithm of Edmonds depends upon translating a problem into an algebraic structure called 
a matroid, but the existence of such a translation can be as hard to determine as the existence of a 
greedy algorithm itself. An alternative characterization of greedy algorithms is in terms of dominance 
relations, a well-known algorithmic technique used to prune search spaces. We demonstrate a pro- 
cess by which dominance relations can be methodically derived for a number of greedy algorithms, 
including activity selection, and prefix-free codes. By incorporating our approach into an existing 
framework for algorithm synthesis, we demonstrate that it could be the basis for an effective engi- 
neering method for greedy algorithms. We also compare our approach with other characterizations 
of greedy algorithms. 


1 Introduction 

A greedy algorithm repeatedly makes a locally optimal choice. For some problems this can efficiently 
lead to a globally optimal solution. Edmonds [Edm71] characterized greedy algorithms in terms of 
matroids. In 1981, Korte and Lovasz generalized matroids to define greedoids [KLS91], The question of 
whether a greedy algorithm exists for a particular' problem reduces to whether there exists a translation 
of the problem into a matroid/greedoid. However, the characterization does not provide any guidance 
on how to construct this translation. In addition, there are problems that have greedy solutions, such as 
Activity Selection and Prefix-free Codes, [CLRS01], that do not seem to fit within the matroid/greedoid 
model. A number of other attempts have been made to characterize greedy algorithms, [BM93, Cur03, 
Cha95, HMS93] but the challenge in all of these approaches is establishing the conditions required for a 
given problem to meet that characterization. Thus, there has been very little work in helping a developer 
actually construct greedy algorithms. 

An alternative approach to constructing algorithms is to take a very general program schema and 
specialize it with problem-specific information. The result can be a very efficient algorithm for the given 
problem, [SPW95, SW08, NC09], One such class of algorithms. Global Search with Optimality (GSO) 
[Smi88], operates by controlled search, where at each level in the search tree there are a number of 
choices to be explored. We have recently [NSC 10] been working on axiomatically characterizing a class 
of algorithms, called Greedy Global Search (GGS), that specializes GSO, in which this collection of 
choices reduces to a single locally optimal choice, which is the essence of a greedy algorithm. Our 
characterization is based on dominance relations [BS74], a well-known technique for pruning search 
spaces. However, this still leaves open the issue of deriving a greedy dominance relation for a given 
problem, which is what we address in this paper. We start with a specialized form of the dominance 
relation in [NSC 10] which is easier to work with. Our contribution is to introduce a tactic which enables 
the two forms of dominance to be combined and also show how the main greediness axiom of GGS 
theory can be constructively applied. We have used this approach to derive greedy solutions to a number 
of problems, a couple of which are shown in this paper. 

Although our derivations are currently done by hand, we have expressed them calculationally as 
we hope to eventually provide mechanical assistance for carrying them out. In addition to providing a 
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Algorithm 1 Program Schema for GGS Theory 

--given x:D satisfying i returns optimal (wrt . cost fn c) z:R satisfying o(x,z) 
function solve : : D -> {Re- 
solve x = 

if <J>(x,rb(x) A i(x)) then (gsolve x rb(x) O) else O 

function gsolve : : D -> {R} -> {R} -> {R} 
gsolve x space soln = 

let gsubs = {s | sGsubspaces x space A Vss G subspaces x space , s 8 X ss} 
soln 5 = opt c (soln U{z I ^(z, space) A o(x,z)}) 
in if gsubs = O then soln 5 

else let greedy = arbPick gsubs in gsolve x greedy soln 5 

function opt :: ((D,R) -> C) -> {/?}-> {/?} 
opt c {s} = {s} 

opt c {s,t} = if c (x , s) >c (x , t) then {s} else {t} 
function subspaces : : D -> R-> { R} 
subspaces x r = {s': ^< x ?A<i>(x,?)} 


process for a developer to systematically construct greedy algorithms, we also believe that our approach 
has a potential pedagogical contribution. To that end, we show that, at least in the examples we consider, 
our derivations arc not only more systematic but also more concise than is found in algorithms textbooks. 

2 Background 

2.1 Greedy Global Search (GGS) Theory 

GSO, the parent class of GGS, is axiomatically characterized in [Smi88], The definition contains a 
number of type and abstract operators, which must be instantiated with problem specific information. 
GGS [NSC 10] specializes GSO with an additional operator, and axioms. The operators of GGS theory, 
which we informally describe here, arc named ro,X, €, <, 5,<F along with an additional type R ; they 
parametrize the program schema associated with the GGS class (Alg. 1). D. R.C.o and c come from the 
problem specification which is described in Section 2.2. 

Given a space of candidate solutions (also called a partial solution ) to a given problem (some of 
which may not be optimal or even correct), a GGS program partitions the space into subspaces (a process 
known as splitting) as determined by a subspace relation <*• Of those subspaces that pass a filter (a 
predicate which is some weakened efficiently evaluable form of the correctness condition, o ) one 
subspace is greedily chosen, as determined by a dominance relation 8 X , and recursively searched 1 . If 
a predicate % on the space is satisfied, a solution is extracted from it. If that solution is correct it is 
compared with the best solution found so far, using the cost function c. The process terminates when no 
space can be further partitioned. The stalling point is an initial space, computed by a function % known 
to contain all possible solutions to the given problem. The result, if any, is an optimal solution to the 
problem. Because spaces can be very large, even infinite, they are rarely represented extensionally, but 
instead by a descriptor of some type R. A relation <E determines whether a given solution is contained in 
a space. 

The process of algorithm development using this theory consists of the following steps: 

1. Formally specify the problem. Instantiate the types of GGS theory. 

2. Develop a domain theory (usually associativity and distributivity laws) for the problem. 


1 Such an approach is also the basis of branch-and-bound algorithms, common in AI 
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3. Instantiate the abstract search-control operators in the program schema. This is often done by a 
mechanically-assisted constructive theorem proving process called calculation that draws on the 
domain theory formulated in the previous step 

4. Apply further refinements to the program such as finite differencing, context simplification, partial 
evaluation, and datatype refinement to arrive at an efficient (functional) program. 

Our focus in this paper is on Step 3, in particular' how to derive the dominance relation, but we also 
illustrate Steps 1 and 2. Step 4 is not the subject of this paper. Specware [S], a tool from Kestrel 
Institute, provides support for carrying out such correctness preserving program transformations. Details 
can be found in [Smi90]. 

2.2 Specifications 

A problem specification (Step 1) is a 6-tuple (D,R,C,/,o,c), where D is an input type (the type of the 
problem instance data), R an output type (the type of the result), C a cost type, i: D —> Boolean is pre- 
condition defining what constitutes a valid input, o : D x R —> Boolean is an output or post condition 
characterizing the relationship between valid inputs and valid outputs. The intent is that an algorithm 
for solving this problem will take any input x : D that satisfies i and return a solution z : R that satisfies 
o (making it & feasible solution) for the given x. Finally c : D x R — > C is a cost criterion that the result 
must minimize. When unspecified, C defaults to Nat and i to true. A constraint satisfaction problem is 
one in which D specifies a set of variables and a value set for each variable, R a finite map from values 
to variables, and o requires at least that each of the variables be assigned a value from its given value-set 
in a way that satisfies some constraint. 

2.3 Dominance Relations 

A dominance relation provides a way of comparing two spaces in order to show that one will always 
have a cheaper best solution than the second. The first one is said to dominate the second, and the second 
can be eliminated from the search. Dominance relations have a long history in operations research, 
[BS74, Iba77], For our puiposes, let £ be a partial solution in some type of partial solutions R , and let 
£0 e he a partial solution obtained by “extending” the partial solution with some extension e : t for some 
problem-specific type t using an operator 0 : R x t — > R. The operator 0 has some problem-specific 
definition satisfying V£-ze£0<?=^ze£. Lift o up to R by defining o(x,z) = Bz • x{z,z) A o(x,z). 
Similarly, lift c by defining c(x.z) = c(x,z ) exactly when 3\z ■ 0(z,£). Then 

Definition 1. Dominance is a relation 8 CD x R 2 such that: 

Vx,£,£ • 8(x,z,z!) (\/e ■ o(x,z! 0c') 0 - Be ■ o(x,z®e) A c(x,£0<?) < c{x,z ®e')) 

Thus, 8(x,z,z!) is sufficient to ensure that £ will always lead to at least one feasible solution cheaper 
than any feasible solution in z! . For readability, 8(x,z ,£) is often written £<5 X £ ■ Because dominance in 
its most general form is difficult to demonstrate, we have defined a stronger form of dominance which is 
easier to derive. This stronger form of dominance is based on two additional concepts: Semi-congruence 
and extension dominance, which are now defined. 

• • ' — D 

Definition 2. Semi-Congruence is a relation -wC DxR~ such that 

\/x,\/e,z,z! ■ (x,£,£) => o{x,z!®e) => o(x,z®e) 

That is, semi-congruence ensures that any feasible extension of z! is also a feasible extension of £ For 
readability, — > {xfzfz') is written £~+ x £. 


Proceedings of NFM 2010, April 13-15, 2010, Washington D.C., USA. 


99 



Synthesis Of Greedy Algorithms Using Dominance Relations 


Nedunuri,Smith,Cook 


And 

Definition 3. Extension Dominance is a relation 8 CD x R 2 such that 

Vx,e,z,'z ■ 8(x,z,z)' => o(x,z®e) A o{x,z 0«)=> c(x,z(Be) < c(x,z! ®e) 

That is, extension dominance ensures that one feasible completion of a partial solution is no more expen- 
sive than the same feasible completion of another part ial solution. For readability, 8(x,z,f) is written 
zSxz'. Note that both and 8 X arc pre-orders. The following theorem and proposition show how the 
two concepts arc combined. 

Theorem 2.1. Let c* denote the cost of the best feasible solution in a space. If ^ is a semi-congruence 
relation, and 8 is an extension dominance relation, then 

Vx,z,? ■ z8fz! A z ~^> x ? => c*(x,z) < c*(x,z!) 


Proof See Appendix □ 

It is not difficult to see that 5 r n is a dominance relation.The following proposition allows us 
to quickly get an extension dominance relation for many problems. We assume we can apply the cost 
function to partial solutions. 

Proposition 1. If the cost domain C is a numeric domain ( such as Integer or Real) and c(x,z(Be) can 
be expressed as c(x,z) + k(x, e) for some functions c and k then 8 X where z8 x z' = c(x,z) < c(x,z!) is an 
extension dominance relation. 

Proof. See Appendix □ 

In addition to the dominance requirement from Theorem 2.1, there is an additional condition on 8, 
[NSC 10]: 

i(x) A (3z €E r ■ o(x,z)) 

(3z* ■ e(z* ,r) Ao(x,z*) Ac(x,z*) = c*(?))V3S* < x ?, Vs< x ? ■ J* S x s~ (2.1) 

This states that, assuming a valid input x, an optimal feasible solution z* in a space r that contains feasible 
solutions must be immediately extractable or a subspace s* of r must dominate all the subspaces of r. 

2.4 Notation 

The following notation is used throughout the paper: i— r is to be read as “instantiates to”. A type decla- 
ration of the form {a : T,b : U, ■ ■ ■ } where T and U arc types denotes a product type in which the fields 
arc accessed by a. b. . . . using a “dot” notation o.a,o.b, etc. An instance of this type can be constructed 
by {a = v, b = w, . . .} where v, w, ■ ■ ■ arc values of type T,U , • • • resp. The notation o { Uj = v, aj = vv, . . .} 
denotes the object identical to o except field a,- has the value v, aj has w, etc. [T] is the type of lists of 
elements of type T, as, accesses the z'th element of a list as, [a] constructs a singleton list with the element 
a, [a | as] creates a list in which the element a is prefixed onto the list as, and as-H-bs is the concatenation 
of lists as and bs, as — bs is the list resulting from removing from as all elements that occur in bs. first 
and last arc defined so that for a non-empty list as — first (as) -FF [last (as)]. Similarly, {T } is the type of 
sets of elements of type T. T >-» U is the type of finite maps from T to U. 


Proceedings of NFM 2010, April 13-15, 2010, Washington D.C., USA. 


100 



Synthesis Of Greedy Algorithms Using Dominance Relations 


Nedunuri,Smith,Cook 


3 A Process For Deriving Greedy Algorithms 

We first illustrate the process of calculating and 8 by reasoning backwards from their definitions on a 
simple example. 

Example 1. Activity Selection Problem [CLRS01] 

Suppose we have a set S = {a,, 02 , ■ ■ ■ , a,,} of n proposed activities that wish to use a resource, 
such as a lecture hall, which can be used by only one activity at a time. Each activity a, has a 
start time s, and finish time /,• where 0 < Sj < fi < 00 . If selected, activity a, takes place in the 
half-open time interval [.y ,■,/)). Activities a, and ay arc compatible if the intervals [.v, , f, ) and 
[s j , fj ) do not overlap. The activity selection problem is to select a maximum-size subset of 
mutually compatible activities. 

Step 1 is to formally specify the problem. The input is a set of activities and a solution is subset of that 
set. Every activity is uniquely identified by an id and a start time ( s ) and finish time (/). The output 
condition requires that activities must be chosen from the input set, and that no two activities overlap. 
For convenience we define a precedence operator A : 

D 1 — > {Activity} 

Activity = {id : Nat,s : Nat,f : Nat} 

R 1 — > {Activity} 
o 1— r A(x,z) • noOvp(x.z) Az C x 

noOvp{x,z ) = V/,y £ z • i / j =k i A j V j A i 
i A ./' = i.f < j.s 
c A(x,z) • ||z|| 

In order to devise a greedy algorithm, the question is what should the basis be for an optimal local choice? 
Should we always pick the activity that starts first? Or the activity that stalls last? Or the activity that 
overlaps the least number of other activities? Or something else? We now show how to systematically 
arrive at the answer. 

Most of the types and operators of GGS theory arc straightforward to instantiate. We will just set 
R to be the same as R. The initial space is just the empty set. The subspace relation < splits a space 
by selecting an unchosen activity if one exists and adding it to the existing partial solution. The extract 
predicate % can extract a solution at any time: 


1 — r R 
1 — r Ax • 0 

i-> A(x,z,z / ) • 3a e x — z ■ z! = zU {a} 

A (z,z) • z = z 
(-»• ? 

? 

The tricky bit is finding bindings for 8 and to complete the instantiation, which we will do in step 3. 
First, in step 2 we explore the problem and try and formulate a domain theory. The composition operator 
© is just U. The A relation can be lifted up to sets of activities by defining the start and finish times of a 
set of activities, namely (it© v)./ = max{w./,v./} and (it® v).j = min{«..v. v. .v } . The following theorem 
will come in handy: 

Theorem 3.1. noOvp(s ) . . ..q, C s ■ s = Ui<;<„JiA(Vi • 1 < i < n s,- A s- l+ \ A noOvp(sj)) A 

noOvp{s„) 


R 

ro 

< 

X 

8 
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o(x,y0 {a} ©e) 

= {defn} 

noOvp(y\J {a} U e) Ay U {a} UeCr 
<= {Theorem 3.1 above} 

y A {a} A e A noOvply) A noOvp(e ) Ay U {a} Lie Cx 
<= {noOvp(y) A noOvp(e) Ay U e C x by assumption} 
y A {«} A e A a £ x 
<= {y A {a'} A e, apply transitivity} 
y A {a} An./ < a 1 .f A a £ x 


Figure 3.1: Derivation of semi-congruence relation for Activity Selection 


This says that any set of non-overlapping activities can be partitioned into internally non-overlapping 
subsets that follow each other serially. 

For Step 3, we instantiate Definition 2 and reason backwards from its consequent, while assuming the 
antecedent. Each step of the derivation is accompanied by a hint (in {}) that justifies the step. Additional 
assumptions made along the way form the required semi-congruence condition. First note that a solution 
z! 0 can be expressed as vLJ { a ! } IJ e or alternatively y®{a'}@e, for som ey,a' ,e, such that, by Theorem 
3.i,5^KHe. Now consider the feasibility of a solution y© { a } © e, obtained by switching out a' for 
some a , assuming o(x,y 0 {(i'}0e) as shown in Fig. 3.1 

That is, y 0 a can be feasibly extended with the same feasible set of activities as y®a! provided y 
finishes before a stalls and a finishes before a' finishes and a is legal. By letting y0« and y 0a' be 
subspaces following a split of y this forms a semi-congruence condition between y 0 a and y 0 a! . Since 
c is a distributive cost function, and all subspaces of a given space arc the same size, the dominance 
relation equals the semi-congruence relation, by Proposition 1. Next, instantiating condition 2.1, we 
need to show that if y contains feasible solutions, then in the case that any solution immediately extracted 
from y is not optimal, (ie. the optimal lies in a subspace of y) there is always a subspace yd) a that 
dominates every extension of y. Unfortunately, the dominance condition derived is too strong to be able 
to establish the instantiation of 2. 1 . Logically, what we established is a sufficient dominance test. That 
is y A {a} A a.f < a'.f A a £ x =$■ y 0 {«} (by© {a'}- Flow can we weaken it? This is where the filter 
comes in. The following theorem shows how to construct a simple dominance relation from a filter: 

Theorem 3.2. Given a filter satisfying V? ■ (3z £ z! ■ o(x,z)) =A <P(x,z!), -'<t>(x,z r ) => Vz ■ z8 x f. 

The theorem says that a space that does not pass the necessary filter is dominated by any space. On 
a subspace y(Ba', one such filter (that can be mechanically derived by a tool such as KIDS [Smi90]) is 
y A {a'}. Now, we can combine both dominance tests with the 1st order valiant of the rule (/; =>rAq=> 
r) (p V q =d r ) , reasoning backwards as we did above as shown in Fig. 3.2. 

The binding for m above shows that 2.1 is satisfied by picking an activity in x — y with the earliest fin- 
ish time, after overlapping activities have been filtered out. Note how in verifying 2.1 we have extracted 
a witness which is the greedy choice. This pattern of witness finding is common across many examples. 
The program schema in Alg. 1 can now be instantiated into a greedy solution to the Activity Selection 
Problem. 

In contrast to our derivation, the solution presented in [CLRS01] starts off by assuming the tasks arc 
sorted in order of finishing time. Only after reading the pseudocode and a proof is the reason for this 
clear (though how to have thought of it a priori is still not!). For us, the condition falls out of the process 
of investigating a possible dominance relation. Note that had we partitioned the solution y0 {a'} © e 
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3a £ x — yf/a 1 £ x — y ■ y 0 a 8 x y © a' 

<= {y A {a} An./ < a'./Aa £ x => y © {a} <5 x y © {a'} &y ^ {a'} =>• y © {a} <5 v y© {a'}} 
3a ex — y,Va' £ x — y • y ^ {a'} V (y A {a} An./ < n 7 ./) 

= {logic} 

3n £ x — y • y A {n} A Vn 7 £ x — y • y A {n 7 } =>■ n./ < n 7 ./ 

= {logic} 

3n £ x — y • y A {n} A Vn 7 £ x — y n {ft | y A {ft}} • a.f < a! .f 
= {define n < ft = a.f < ft./} 

3n £ x — y • y A {n} A Vn 7 £ x — y n {ft | y A {ft}} • m < a’ =>■ a.f < a! .f 
where m = min< x — y n {ft | y A {ft}} 

= {law for monotone /; : (\/x £ S • m < x /;(a')) = /;(/??) } 

3n £ x — y • y A {n} An./ < m.f where m = min<x — y(~l {ft | y A {ft}} 

= {law for anti-monotone p : (3jc £ A ■ in < x A /?(*)) = /;(/??) } 
m.f < m.f where m = min< jc — yfl {ft | y A {ft}} 

true 


Figure 3.2: 


differently as r ^ {n 7 } A y, we would have aiTived at an another algorithm that grows the result going 
backwards rather than forwards, which is an alternative to the solution described in [CLRS01]. 

Next we apply our process to the derivation of a solution to a fairly non-trivial problem, that of 
determining optimum prefix-free codes. 

Example 2. Prefix-Free Codes 

Devise an encoding, as a binary string, for each of the characters in a given text file so as to 
minimize the overall size of the file. For ease of decoding, the code is required to be prefix- 
free , that is no encoding of a character is the prefix of the encoding of another character (e.g. 
assigning “0” to ’a’ and “01” to ’b’ would not be allowed). 

D.A. Huffman devised an efficient greedy algorithm for this in 1952. We show how it can be system- 
atically derived. Step 1 is to specify the problem. The input is a table of character frequencies, and the 
result is a table of bit strings, one for each character in the input, satisfying the prefix free property. 

D i— > Char Frequency 

Char = Frequency = Nat 
R i— > Char >— » [ Boolean ] 

o i— > Xx,z. dom(z) = dom(x ) A Vc / c' £ dom(z ) • ^ prefixOf (z.{ c ) , z ( c 7 ) ) 
prefixOf(s,t) = 3u ■ t = s++u V s = t-\-+u 
C kx,Z. I cedom(z) lk( c )l! x *( c ) 

Often a good way of ensuring a condition is to fold it into a data structure. Then the rules for 
constructing that data structure ensure that the condition is automatically satisfied. It turns out that a 
binary free in which the leaves arc the letters, and the path from the root to the leaf provides the code for 
that letter, ensures that the resulting codes arc automatically prefix-free 2 . One obvious way to construct 


2 It is possible to systematically derive this datatype by starting with an obvious datatype such as a set of binary strings and 
folding in a required property such as prefix-freeness. However, we do not pursue that here 
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Extension, e 

Partial 

solution 

[s,w,t,u,v] 


Figure 3.3: An extension applied to a partial solution with 5 trees 

such a tree is by merging together smaller trees. This will be the basis of the < relation: the different 
subspaces arise from the t(t — l)/2 choices of pairs of trees to merge, where t is the number of trees at 
any point in the search. The starting point To is the ordered collection of leaves representing the letters. 
The extract predicate % checks there is only one tree left and generates the path to each leaf (character) 
of the tree, which becomes the code for that character. With this information, we can instantiate the 
remaining terms in GGS, except for 8 ((} is the non-elementaryZ?/«7>ee constructor) 

R i— > \BinT ree] 

?o i— ► Ax • asList(dom(x )) 

< !->■ A(x,z,£) • 3s, t e££ = [(s,t) | (z — s-t)] 
x A (z,z) • ||z|| = 1 A Vp€ pciths(z ) ■ z(last(p )) = first (p ) 

paths((s,t )) = (map prefixO paths(s ) ) +-\-(map prefix! paths(t )) 
paths (l ) = [/] 

prefixO(p) = [0 | p\, prefixl(p) = [1 | p\ 

<f> i — true 

8 i-> ? 

One interesting difference between this problem and the Activity Selection problem is that every 
subspace leads to a feasible solution. For that reason, is just true. It is not obvious what the crite- 
rion should be for a greedy choice. Should it be to combine the trees with the least number of leaves, 
or those with the least height, or something else? To proceed with our process and apply Defs. 2 and 
3, we will define © as a left-associative binary operator whose net effect is to merge two BinTree s 
from its left argument together into another BinTree. The right argument specifies which trees are to 
be merged. That is, £ © (/. /) = [(£,£/) | z~Zi — £/] . For example, Fig 3.3 shows the merger of trees 
s and t and the merger of trees u and v in a partial solution z to form two subtrees with roots a and b. 
This is described by the expression (£© (1,3)) © (3,4) = £© (1,3) © (3,4). The extension in this case is 
(1,3) © (3,4). A semi-congruence condition is shown in Fig. 3.4. Assuming o(x,£ ffie) and a definition 
of lvs{z) = maplast (paths (z)). (Note we omit the remainder of the output condition since it is implied by 
the Binary Tree construction) This says that any two partial solutions of the same size are extensible with 
the same feasible extension. This semi-congruence condition is trivially satisfied by any two subspaces 
of a split. For the extension dominance condition, it is easy to show that c(x(z®e) can be expressed as 
c(x,£) + k(x,e) for some k where c(x,z) = t!!=i ®^(z)(i) • x(lvs(z)i ) where d(z) is a function returning 
the depth of its argument leaf within the tree z, and therefore by Prop. 1, it is sufficient to determine 
conditions under which c(x.z) < c(x,£). However, if we try to calculate such a condition as we have 
done for semi-congruence we will end up with an expression that involves the depths of individual leaves 
in the trees. Is there a simpler form? We have been investigating ways to provide a developer with hints 
about how to proceed. We call these tactics. In earlier work [NSC09] we introduced tactics for the 
derivation of operators for non-optimization problems. We now introduce a tactic for dominance rela- 
tions. We have used this tactic to derive greedy solutions for a number of problems, including Machine 
Scheduling, several valuations on the Maximum Segment Sum Problem, [NC09], Minimum Spanning 
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o(x,z®e) 

= {defn of o on R} 

3z • x{ZiZ® e ) Ro(x,z) 

= {defn of x,oj 

3z « ||?©e|| = 1 A Vp E paths(z(Be ) • z(l a st(p)) = first (p) Adom(z) = xA--- 
= {intro defn} 

||z©e|| = 1 A dom(z) =x A ■ ■ ■ 

where z = {last(p) i-> first(p) \ p E paths(z(B e)} 

<= {o(x,z! ®e) =>• dom(z') = x where z = {last(p) i— >• first(p) \ p E paths{z! © e)}} 
||z©e|| = 1 A asSet(lvs(z)) = asSet (lvs(z!)) 

= {split does not alter set of leaves} 

||z©e|| = 1 

= {\\z®e\\ = ||z1| - ||e|| . ||z'©e|| = 1} 



Figure 3.4: Derivation of extension dominance relation for Huffman problem 

c(z) < c(?) 

{ unfold defn of c} 

l1=i ( ' S) I (d{s)(i) + h) ■x(lvs(s)i) + Y!!lf^(d(t)(i) + h) ■ x{lvs(t)i ) 

+ L|= 1 ( ")I (^(«)(0 + 2) -x{lvs(u)i) + (d(v)(i) + 2) • x(lvs{v)i ) 

< 

l1=i ( ")I(^( m )(*) + / j ) -x(lvs(u)i) + l|=i (v)I (^(v)(0 + /j) • x(lvs(v)i ) 

+ L!=f)'(dW(i) + 2) -x(lvs(s)i) + l!=i WI (^(0(0 + 2 ) • x{lvs{t)i ) 

{algebra} 

(h - 2) • x(lvs(s)i ) + (. h - 2) • e£ (,)I x(lvs(t)i ) 

< (/ 7 . - 2) • x(lvs(u)i ) + (/ 7 . - 2) • lf = ra i (v)l x(lvs(v)i) 

{algebra} 

^|Zvi(i)| x(lvs(t^i) < e!=i^^(^v5(m)) +L/=f V) ^( ; v5(v),-) A/j > 2 


Figure 3.5: 


Tree, and Professor Midas’ Driving Problem. 

Exchange Tactic: Try to derive fl dominance relation by comparing a partial solution ?© aT) (X T)b (as- 
suming some appropriate parenthesization of the expression) to a variant obtained by exchanging a pair 
of terms, that is, y®b@(X®a, with the same parenthesization 

Given a partial solution y, suppose frees s and t arc merged first, and at some later point the free con- 
taining .s' and t is merged with a free formed from merging u and v (tree satcubv in Fig. 3.3), forming 
a partial solution z. Applying the exchange tactic, when is this better than a partial solution z! resulting 
from swapping the mergers in z, ie merging u and v first and then .v and tl Let d{T)j be the depth of leaf 
i in a tree T, and let h be the depth of s (resp. u) from the grandparent of u (resp. s) in z (resp z!), (the 
distance from c to the root of s in Fig. 3.3). The derivation of the extension dominance relation is shown 
in Fig. 3.5. 

That is, if the sum of the frequencies of the leaves of s and t is no greater than the sum of the 
frequencies of leaves of u and v then s and t should be merged before u and v. (The condition h > 2 
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simply means that no dominance relation holds when (u,v) is immediately merged with (s,t). It is clear 
in that case that the tree is balanced). How does this help us derive a dominance relation between two 
subspaces after a split? The following theorem shows that the above condition serves as a dominance 
relation between the two subspaces [(s,t) \ (y — s — f)] and [(u,v) \ (y — u — v)]: 

Theorem 3.3. Given a GGS theory for a constraint satisfaction problem, (3 a • (y 0 a -® (X © h) 8 X (y 0 
fc®a$a)) =^y© flr 8 x y®b 

By using a witness finding technique to verify condition 2.1 as we did for Activity Selection, we 
will find that the greedy choice is just the pair of trees whose sums of letter frequencies is the least. 
This is the same criterion used by Huffman’s algorithm. Of course, for efficiency, in the standard al- 
gorithm, a stronger dominance test is used: x (lvs(s)i) < Y}j=\“^ x(lvs(u)i) ^ x(lvs(t)t) < 

l!=i (v)l *(Mv)i) and the sums arc maintained at the roots of the frees as the algorithm progresses. We 
would automatically arrive at a similar procedure after applying finite differencing transforms, [Smi90, 
NC09]. In contrast to our stepwise derivation, in most presentations of Huffman’s algorithm, (e.g. 
[CLRS01]) the solution is presented first, followed by an explanation of the pseudocode, and then several 
pages of lemmas and theorems justifying the correctness of the algorithm. The drawback of the conven- 
tional approach is that the insights that went into the original algorithm development arc lost, and have to 
be reconstructed when valiants of the problem arise. A process for greedy algorithm development, such 
the one we have proposed here, is intended to remedy that problem. 

4 Related Work 

Curtis [Cur03] has a classification scheme for greedy algorithms. Each class has a some conditions that 
must be met for a given algorithm to belong to that class. The greedy algorithm is then automatically 
correct and optimal. Unlike Curtis, we arc not attempting a classification scheme. Our goal is to simplify 
the process of creating greedy algorithms. For that reason, we present derivations in a calculational style 
whenever the exposition is clear. In contrast, Curtis derives the “meta-level” proofs, namely that the 
conditions attached to a given algorithm class in the hierarchy arc indeed correct, calculationally but the 
“object-level” proofs, namely those showing a given problem formulation does indeed meet those con- 
ditions, arc done informally. We believe that this should be the other way around. The meta-level proofs 
arc (hopefully) carried out only a few times and arc checked by many, but the object level proofs arc car- 
ried out by individual developers, and arc therefore the ones which ought to be done calculationally, not 
only to keep the developer from making mistakes but also with a view to providing mechanical assistance 
(as was done in KIDS, a predecessor of Specware). Another difference between our work and Curtis is 
that while Curtis’s work is targeted specifically at greedy algorithms, for us greedy algorithms arc just a 
special case of a more general problem of deriving effective global search algorithms. In the case that the 
dominance relation really does not lead to a singleton choice at each split, it can still prove to be highly 
effective. This was recently demonstrated on some Segment Sum problems we looked at, [NC09]. Al- 
though the dominance relation we derived for those problem did not reduce to a greedy choice, it was 
nonetheless key to reducing the complexity of the search (the width of the search tree was kept constant) 
and led to a very efficient breadth-first solution that was much faster than comparable solutions derived 
by program transformation. 

Another approach has been taken by Bird and de Moor [BM93] who show that under certain con- 
ditions a dynamic programming algorithm simplifies into a greedy algorithm. Our characterization in 
[NSC 10] can be considered an analogous specialization of (a form of) branch-and-bound. The difference 
is that we do not require calculation of the entire program, but specific operators, which is a less onerous 
task. 
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Helman [Hel89] devised a framework that unified branch-and-bound and dynamic programming. 
The framework also incorporated dominance relations. However, Helman's goal was the unification of 
the two paradigms, and not the process by which algorithms can be calculated. In fact the unification, 
though providing a very important insight that the two paradigms arc related at a higher level, arguably 
makes the derivation of particular algorithms harder. 

Charlier [Cha95], also building on Smith’s work, proposed a new algorithm class for greedy algo- 
rithms that embodied the matroid axioms. Using this class, he was able to synthesize Kruskal’s MST 
algorithm and a solution to the 1 / 1 / £ 7} scheduling problem. However he reported difficulty with the 
equivalent of the Augmentation (also called Exchange) axiom. The difficulty with a new algorithm class 
is often the lack of a repeatable process for synthesizing algorithms in that class, and this would appear 
to be what Charlier ran up against. In contrast, we build on top of the GSO class, adding only what is 
necessary for our purposes. As a result we can handle a wider class of algorithms than would belong in 
Chari icr’s Greedy class, such as Prim’s and Huffman’s. 
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5 Appendix: Proofs of Theorems 

5.1 Proofs of Theorem 2.1 and Proposition 1: 

Theorem 2.1: If ~ ■> is a semi-congruence relation, and <5 is a extension dominance relation, then 

\/x,Vz,z! ■ z8 x z A z~^> x z =$■ c*(x,z) < c*(x,t) 

Proof. By contradiction, (input argument x dropped for readability). Suppose that z8fz! A z ~^ x f but 
3z'* G zf, Off*) A c(z!*) < c*(z), that is c{z!*) < c(z) for any feasible z G z ■ We can write z!* as z! ®e for 
some e. Since z'* is cheaper than any feasible zGz, specifically it is cheaper than z = z ( l) e, which by the 
semi-congruence assumption and Definition 2, is feasible. But by the extension dominance assumption, 
and Definition 3, this means c(z) < c(z!*), contradicting the initial assumption. □ 

Proposition 1: If the cost domain C is a numeric domain (such as Integer or Real) and c(x,z©e) 
can be expressed as c(x,z) +k(x,e) for some functions c and k then 8 X where z8 x z' = c(x.z) < c(a'.F) is 
an extension dominance relation 

Proof. By showing that Definition 3 is satisfied. c(f(Be) < c(f ®e) = 8(z) +k(e) < c(f ) +k(e) by 
distributivity of c which is just c(z) < c(f) after arithmetic. □ 
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